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ABSTRACT 

The audit report (1970-71) on the Bilingual Education 
Program of the Harlandale and San Marcos Independent School Districts 
gives a critique on the quality of the project evaluation^ discusses 
the comparative findings of the project evaluation arid the audits and 
confinns or questions the need for program modifications proposed in 
the evaluation. The audit report is divided into 5 sections: (1) 
introductory and general comments on the quality of the prpject 
evaluation and the comparative findings of the project evaluation and 
the audit; (2) detailed critique of the product^ process, and 
management evaluation for each component, based on an assessment of 
the instruments used, data collection procedures, data analysis 
techniques, and data analysis pi sentation; (3) description pf the 
auditor's on-site findings and their correlation with the evaluator* s 
data and reports on a component by component basis, summary of 
consistencies and discrepancies, and interpretation of discrepancies; 
(4) recommendations for evaluation design revisions with a rationale 
for each recommendation; and (5) confirmation or questioning of the 
need for program modifications proposed in the project evaluation^ 
(Author/NQ) 
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EDUCATIONAL ACCOMPLISHMENT AUDIT 
REPORT 



Purpose of the Report I 
This Audit Report is made in compliance with the contractual 
agreements, legal prescriptions, and official directives under the 
Provisions of Title VII of Public Law 89-105 as amended, for the 
establishment and operation of bilingual education programs • This is 
an Audit Report on the Bilingual Education Program of the Harlandale 
Independent School District, The San Marcos Independent School District, 
and the Southwest Texas State University, with Dr, B. E. Hughes as 
program director. The purpose of this report is to give a critique 
concerning the quality of the project evaluation and discuss the comparative 
findings of the project evaluation and the audit and to confirm or question 
the need for program modifications which have been proposed as a result of 
project evaluation. 

Scope of the Report 
The Audit Report follows rather closely the suggested Audit Report 
Content Areas received from the United States Office of Education and 
made a part of the contractual agreement between the auditor and the 
program director, as follows: 

1. Introductory and general comments concerning the 
quality of the project evaluation and the comparative 
findings of the project evaluation and the audit. 

2. Detailed critique of the product, process, and management 
evaluation conducted for each component, based on an 
assessment .of the instruments used, data collection procedures, 
data analysis techniques, and data analysis presentation. 



3. Description of the auditor's on-site visit findings and 
their correlation with the evaluator's data and reports, 
on a component by component basis; sxnnmary of consistencies 
and discrepancies, and interpretation of the discrepancies. 

4* Recommendations for revision of the evaluation design, 
including a rationale for each recommendation. Since 
the auditor's objectivity can be retained only if the , 
selection of a specific corrective action is a local 
decision, he should provide general rather than specific 
recommendations, posing several alternative actions or 
possible sources of assistance to the LEA in correcting 
the deficiency* 

5. Confirmation or questioning of the need for program 
modifications which have been proposed as a result of 
project evaluation. 

Each of the audit areas described above will be referred to as 

sections with the section numbers corresponding with the audit area 

description numbers. 

SECTION I 

General Comments Concerning the Quality 
of the Project Evaluation and 

Comparative Findings • 

The Evaluation Report for the School Year 1970-71 follows closely 
the kinds of evaluations that were listed with the objectives for process 
and performance. Generally speaking the evaluation report is quite 
adequate and comprehensive. Many of the suggestions that were made in 
the pre-audit report have been made a part of the revised evaluation design 
and encon?)assed In the evaluation report. Specifically these include 
demographic data for the two districts, baseline data for the first grade 
children in linguistic competence and general ability, and measures of 
skills in Spanish in the first grade. 
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Even though the data are analyzed very well to reveal the extent of 
the attainment of the specified learning objectives, perhaps additional 
information could have been extracted from the data for more specific 
instructional analysis for the further improvement of learning. Specific 
reference is made here to intra-class and inter-class variability and 
types of information or skills in which the most progress was made or in 
which the greatest deficits occurred. However, it must be quickly pointed 
out that in many instances this progress or lack of progress was frankly 
reported. 

The evaluator has managed to maintain a high degree of objectivity. 
The auditor has detected no bias that is due to subjectivity or a myopic 
view of the project. In some instances additional emphasis could have 
been placed on the successes or indications of progress of the bilingual 
program without distorting accuracy of reporting. 

The evaluation report covers the first four components of the program 
rather comprehensively but is almost totally silent on the Project 
Coordination Component except for the inclusion of an organizational 
chart for the Bilingual Consortium. The other four conpo: :nts are: 
(1) Curriculum Development Component; (2) Instructional Component; 
(3) Teaching Improvement Component; (4) Community Involvement Component. 

Most of these general statements will be supported with specifics 
in the next section. 
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SECTION II 

Detailed Critique of the Product, Process, and 
Management Evaluation Based on an Assessment 
of the ^Ir^truments Used, Data Collection 
Procedutes, Data Analysis Techniques, 
and Data Analysis Presentation 

The evaluation report begins with a rather accurate but brief 
Summary of the evaluation. The second paragraph of this summary 
relterat^^s "the primary objective of the program is to prevent educational 
retardation of these children by teaching in Spanish while comf Cence in 
English is being developed''. Subsequent data do not substantiate that the 
Spanish language is being used enough to say that it carries the instructional 
load while competence in English is being developed. The basic and most 
significant findings in the evaluation are reported in th3 summary with a 
high degree of accuracy. 

The evaluator has appropriately called attention to many aspects of 
the two school districts involved in the program that are similar and has 
Indentlfled common instructional needs. Demographic data on the two 
districts were collected by the evaluator this year. She summarizes her 
findings by saying, "Demographic data were roughly equivalent for the two 
districts". The auditor has seen enough of the data and asked enough 
questions to be convinced that the evaluator 's statement is correct. 
However, it certainly would have been more effective if these data had 
been clearly presented in a comparative table for all to see and examine. 
Some of the auditor's hypotheses about the differences between the two 
districts were apparently in error. The percentages of Spanish speakers 
and the low- Income levels confirm the need for bilingual instruction. 
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In the Program Description the evaluator makes reference to the 
Organizational Chart, Table II, Page 9. From this chart It can be easily 
seen that the Bilingual Consortium is an administrative overlay on the 
admlnstratlve structure of the two districts. The extent to which an 
administrative overlay may become effectively operative is dependent upon 
the extent of its acceptance and integration with the existing administrative 
structure. The most sensitive area of conflict or confluence is revealed 
in the two solid lines running to nhe bilingual teachers. One of these 
lines comes from the Project Director through the Bilingual Coordinator 
and the other comes from the building principal and the administrative 
officers above htm. Whr.ther these administrative lines are characterized 
by confluence or conflict depends on the extent the bilingual program is 
integrated into the total curricular learning program of the elementary 
schools. This integration can occur through communication, consultation, 
joint planning, common objectives, and complete coordination. If these 
harmonizing factors are absent or inactive » these lines will become 
sources of conflict. The teachers will begin to feci that they are called 
on to carry on the two programs, the regular curricular program under the 
direction of the principal and the bilingual progri.-n under the project 
director. If the Intent of Title VII to develop bilingual programs that 
will continue after Federal funding has ceased is realized, then evidence 
of rhis fully Integrated instructional program should be emerging at the 
end of this second year. The auditor's sampling of more than 30% of the 
teachers and more than 50% of the principals reveals that there is a 
feeling that the teachers are being called on to carry on two separate 
programs. The two district superintendents share responsibility with the 
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project director for the existence of this condition. This emerging 
difficulty should be resolved before it becomes more acute. This is 
a problem of having an integrated Curriculum for the pupils. This will 
be further discussed under the findings concerning teacher attitudes. 

Under Program Descriptions , Pages 6-13 of the Evaluation Report, 
the evaluator has very accurately described what has happened in the 
bilingual program during its second year of operation. The auditor 
considers this an essential aspect* of any evaluation. Factual reporting 
provides the setting for the statistical measures vhich are reported 
later. 

Tlie evaluator on Page 12 of the P rogram Description has stated, 
"The English language arts program began with an oral approach in the 
first grade. Typical language readiness procedures were used, with 
explanations being made in Spanish when necessary". This is realistic 
reporting and far more descriptive of what actually took place than the 
statement in the objectives that instruction was carried on in Spanish 
whil2 conq)etence in English was being developed to which reference was 
made earlier in this report. The evaluator has not indicated any part 
of the program where Spanish carries the total instructional load except 
in Spanish language arts. The auditor's on-site visits confirm this 
viewpoint* 

Putting ability measures back into the evaluation design to provide 
additional baseline data for the first grade seems most appropriate. The 
useful effect of giving the tests with directions in both English and 
Spanish is reflected in the ocore on the Goodenough and Otis scores in 
Table IV. The Otis Non-verbal pre- and post-test indicate more than 
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normal progress in development of ability during the year. This could 
indicate that the bilingual programs in the first grade are eliminating 
some of the deficits in learning that have been apparent. The verbal 
scores were slightly lower than the non-verbal scores. 

The data in regard to the voluntary use of English is interesting, 
but it is impossible to tell whether it reveals anything about the 
efficiency of the program since there is no control group of similar 
pupils who were not in the program. 

The auditor concurs with what the evaluator said about the Metropolitan 
Achievement Tests. The evalnator pointed out the highly significant fact 
that none of the first grade classes attained the objective in reading 
meaning, while 50% of the first grade classes attained the objective in 
word discrimination. This strongly suggests that methodology for teaching 
reading should be reexamined. Recommendation Number 5 in the first 
paragraph of Page 20 is especially important. The teaching of reading 
meaning showed to be a little better in the other grades but this was 
still the lowest area. 

Since the third and fourth grades received separate grades on 
problem solving and concepts, it would have been interesting if the 
evaluator had reported how these scores compared. 

The evaluator has indicated some dissatisfaction with the per- 
formance tests in determining competence in Spanish. A careful 
examination of the performance objectives and the measures that accon^any 
them reveals that these fragmented measures have little to do with the 
holistic competence in Spanish language skills in meaning, understanding, 
and communication. The evaluator has indicated that revisions will be 
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made for next year* The objectives referred to are on Page 28 of 
the evaluatloGi* 

One of the major objectives of the social studies units was to 
give the children more pride in their heritage* An examination of 
these units shows that they were designed to give the pupils know- 
ledge and pride in their American heritage but the true Mexican-American 
heritage is almost totally missing* Even the opportunity to have pupils 
name Texans with Spanish surnames who fought in the Alamo was missed* 
The criteria for measuring performance in the social studies have little 
or no relation to social studies concepts and to the real purposes for 
teaching social studies* The measurements arc of the factual recall 
type in Spanish and English* The data resulting from the application 
of these criterion measures can have little significance* The social 
studies units need to reflect the warm, affective nature of the Mexican- 
American culture* At least 5o% of the measures should be in the affective 
domain* 

The evaluator has pointed out the Increased involvement of students 
at Southwest Texas State University who were interested in becoming 
bilingual elementary teachers* The auditor visited in 10 classrooms in 
the pre-test and in the post-test adminif^tration in both school districts* 
The students, with minor excer:.ions, did an excellent job of administering 
ttie tests correctly* This concurs with the evaluators statements on 
Page 29* 

The evaluator has explained the various facets of the Staff 
Development Component as it operated this year* In addition to pre- 
service training with pre- and post-tests to measure improvement, 



interactional analysis of every classroom with feedback to the teachers, 
video-taping with teacher self analysis, and classroom visits by the 
coordinators. All of these were good techniques and procedures. This 
was an improvement over the first year. 

The teachers were rated twice by the principals and by the coordinators. 
The teachers also rated themselves These ratings were apparently helpful 
but there was a very definite tendency on the part of the principals 
and one coordinator to over-rate the teachers. One principal rated his 
teachers so high in the first rating that it was almost impossible for 
him to show they had improved by the time of the second rating. Some 
additional and new approaches should be used that will enable the true 
feelings of the teachers to be ascertained more accurately and 
communicated more quickly. 

The activities of the Community Involvement Componer:t have increased 
this year as indicated by the evaluator. The evaluation of this component 
is accurate and complete. The auditor ^s findings confirm that of the 
evaluator. 

The evaluator ^s recommendations are logical and in accord with the 
findings of the evaluation. The auditor finds them appropriate and 
acceptable. Hovever, there could easily be lack of agreement in the 
methods and techniques designed to implement the recommendations. For 
example. Recommendation Number 3 indicates that a closer check should 
be kept of the amount of time spent teaching various parts of the 
curriculiim and the amount of time spent teaching in English and in 
Spanish. The implication is that changes could be brought about by 
closer supervision and other external controls. It is very doubtful 



that this would accomplish the gonl* The change In the amount o£ time 
spent In teaching in Spanish would call for a change of attitude on 
the part of teachers* It might be true that this would also necessitate 
a change of attitrde on the part of a number of other people before 
this could be acccnuplished at all. Furthermore, teaching the 
technicalities of language would likely have very little Impact in 
teaching of reading comprehension. 

The recommendations that have been made by the evaluator merit 
careful study by those with leadership responsibilities* They should 
not be ignored. 

SECTION III 

The Auditor's On-Site Visit Findings and Their 
Correlation with the Evaluator *s Data 

The auditor spent two days observing the administration of the 
pre-tests and two days obsen;ing the post-test administration in both 
districts. This involved six schools and 10 classrooms on each occasion. 
The auditor found the testing procedures being followed properly. The 
college students uho administered the tests had been properly instructed 
and did their jobs well* There was some minor interference from over- 
anxious teachers. However, most of the teachers either did not 
participate or worked with the test administrator in a helpful way. 
A five per cent sample of the test scoring in the office of the evaluator 
revealed a remarkably high degree of accuracy. The auditor feels assured 
that the tests were properly administered and accurately scored. 

The auditor spent two days observing bilingual classrooms > interviewing 
teachers and pupils » interviewing principals, and examining classroom 
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materials. The auditor found that some of the teachers were much too 
^ycevned with mechanical drills and rote teaching. Even when exper- 
iences were discussed, many of them were not basic to the children's 
interest. The teachers in general were more concerned with the isolated 
fragments of learning than they were with the meaningful use of language 
as a comunication system. There were very notable exceptions. The 
auditor observed one first grade and one fourth grade where masterful 
teaching was in progress. There was a total range irom masterful 
performance to lifeless perfunctory performance. 

From these on-site visits throughout the year, the auditor gained 
evidence that the amount of teaching in Spaillsh was minimal. Some 
teachers admitted frankly that they did not use more than 30 minutes 
per day for instruction in Spanish including Spanish language arts. Some 
Indicated that they used Spanish throughout the day for clarification 
purposes. One principal indicated that In a team teaching situation, 
one hour was used In the exchange of teachers for instruction in Spanish. 
The auditor timed the exchange on two ocassions. One day it was 40 
minutes and on the next observation it was 30 minutes. The auditor's 
findings merely confirm what the evaluator has said. 

The most important findings of the auditor were in regard to the 
attitudes of the teachers and principals toward some aspects of the 
program. The evaluation does reflect the existence of these attitudes 
but this does not mean that the evaluator and the project director are 
unaware of their existence. The auditor, evaluator, and project 
director have discussed these attitudes extensively. 

ERIC 



The teachers and principals seem to feel that too much time is being 
spent in testing. This was an almost unanimous opinion in Harlandale but 
it was only about 50% prevalent in San Marcos. The auditor does not 
agree with the teachers and principals that there is too much measurement 
being done in the bilingual program itself. However, other measurements 
and testing are being carried on by regular district curriculum programs. 
This is another indication of the existence of dual, uncoordinated 
learning programs. It is recommended that all aspects of the testing 
program be restudied and that steps be taken to coordinate the total 
testiig program of the two districts. 

The teachers feel that they are unduly burdened by having to carry 
on two instructional programs at the same time. They feel that they 
must conform to the regular instructional program of the school districts 
and do their bilingual teaching in addition. This attitude is far more 
prevalent in Harlandale than it is in Saa Marcos. About^50% of the 
teachers in San Marcos feel this way. Two teachers in Harlandale and 
three in San Marcos thought that everything about the entire bilingual 
program was satisfactory. 

The auditor views this as a serious situation. The bilingual program 
will undoubtedly disappear as soon as federal funding is over unless the 
program becomes completely integrated into the total instructional 
program. This means a unified, integrated curriculum. This can only 
be accomplished at the highest levels. 

Future evaluations should include more complete reports on teacher 
attitudes even if it requires anonymous surveys. 
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The coordinators and community liaison workers have made all their 
materials available to the auditor in the regard to the Community 
Involvement Component. These materials, questionnaire results, visits 
to advisory meetings, and talks with parents indicate a high degree of 
favorable community opinion and cooperation. This confirms the 

evaluator's findings. 

The on-site visits with the coordinators » evaluators, and project 
director have been most pleasant and useful. They have made all 
materials available, the data have been readily accessible, and they 
have been frank and open in their opinions and responses to questions. 
This attitude has facilitated the work of the auditor and made feedback 
possible and immediate. 

SECTION IV 

Reconsnendations for Revisions of the Evaluation 

Design 

Some reconmendations for revision of the evaluation design have 
been discussed under the specific critique and the on-site visit findings. 
However they will be summarized here. 

Since one of the original objectives of the program was the prevention 
of academic retardation and enabling the pupils to Increase their 
achievement to the point they would be approximately on grade level, it 
is recommended that the data be more closely related to progress in grade 
level attainment. Comparison with previous years could be included to 
see whether any progress is being made toward closing the gap. Percentile 
reporting does this somewhat. However, using percentiles for computation 
in distributions leaves much to be desired In statistical accuracy. The 
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evaluator has not done this, but she has also \)een limited in showing 
intra-* and inter-group variability • The use of scores and grade level ^ 
equivalents should be explored. 

The exploration of principal and teacher opinions and attitudes 
should be more realistic. These instruments could ask more specific 
and penetrating questions. More opportunities for open-ended responses 
could be provided. Anonymity should not be ruled out as a useful 
technique. However, the auditor thinks that people should be willing 
to take personal responsibility for their attitudes and opinions. 
Frequent frank interviews are useful. The auditor's findings indicate 
that the evaluation does not realistically report teacher and principal 
attitudes . 

An evaluation of the Coordination Component should be included in 
the evaluation. This should include all the relationships between the 
project administrative overlay and the regular administrative relation- 
ships within the districts. As has been previously discussed, this is 
essential to having a smooth functioning, integrated instructional program. 
Making bilingual instruction a permanent part of the school program is 
dependent upon It. 

The evaluation is well written, detailed, and quite accurate. It 
is worthy of careful study by all persons charged with administrative 
r espons ibillty . 
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SECTION V 

Confirming or Questioning of the Need for 
Program Modifications 

Most of the things that should be said in this -section have been 
written in more detail elsewhere already • 

1. The evaluator's recommendations should be studies carefully by 
administrators with a view to their implementation. The methods used 
for their implementation should be in accord with sound educational 
principles. 

2. Administrative action should be taken to deal with the concept 
of a dual learning program and the resulting feeling of an unfair work 
load that is affecting the teachers' attitudes. Curriculum unil:ication 
should be considered. The teachers seem to endorse the concept of 
bilingual education almost unanimously. 

3. This program is probably one of the better bilingual programs. 
Many aspects of the program have improved during the second year's 
operation. This progress should be continued. 

4. The project coordinators have functioned well this year. Both 
of them are highly respected by the teachers. 

5. The addition of a full-time evaluator has been most helpful to 
the prpject. 
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